具有混合精度量化的大DNN可以实现超高压缩，同时保持高分类性能。但是，由于找到了可以引导优化过程的准确度量的挑战，与32位浮点（FP-32）基线相比，这些方法牺牲了显着性能，或者依赖于计算昂贵的迭代培训政策这需要预先训练的基线的可用性。要解决此问题，本文提出了BMPQ，一种使用位梯度来分析层敏感性的训练方法，并产生混合精度量化模型。 BMPQ需要单一的训练迭代，但不需要预先训练的基线。它使用整数线性程序（ILP）来动态调整培训期间层的精度，但经过固定的硬件预算。为了评估BMPQ的功效，我们对CiFar-10，CiFar-100和微小想象数据集的VGG16和Reset18进行了广泛的实验。与基线FP-32型号相比，BMPQ可以产生具有15.4倍的参数比特的模型，精度可忽略不计。与SOTA“在培训期间”相比，混合精确训练方案，我们的模型分别在CiFar-10，CiFar-100和微小想象中分别为2.1倍，2.2倍2.9倍，具有提高的精度高达14.54％。

随着大型预训练的Vison语言模型（如剪辑）的出现，可以通过及时调整来调整可转让表示形式。及时调整试图从存储在预训练的视觉模型的图像和文本编码器中的常识中探索有益信息，以探索下游任务。最近提出的名为“上下文优化”（COP）的方法将一组可学习的向量从语言侧引入文本提示符，而单独调整文本提示符则不会影响图像编码器的计算视觉特征，从而导致了次级优势。在本文中，我们通过学习文本提示并同时为文本和图像编码器提供双重模式提示调整范式。此外，为了使视觉提示更多地集中在目标视觉概念上，我们提出了类感知的视觉及时调整（CAVPT），该调整是通过在模板提示和视觉类别令牌嵌入的语言描述之间进行交叉注意来动态生成的。我们的方法提供了一种新的范式来调整大型预训练的视觉模型，并在8个数据集上进行了广泛的实验结果，证明了该方法的有效性。我们的代码在补充材料中可用。

自动艺术文本生成是一个新兴主题，由于其广泛的应用而受到越来越多的关注。艺术文本可以分别分为三个组成部分，内容，字体和纹理。现有的艺术文本生成模型通常着重于操纵上述组件的一个方面，这是可控的一般艺术文本生成的亚最佳解决方案。为了解决这个问题，我们提出了一种新颖的方法，即Gentext，以通过将字体和纹理样式从不同的源图像迁移到目标图像来实现一般的艺术文本样式转移。具体而言，我们当前的工作分别结合了三个不同的阶段，分别是具有单个强大的编码网络和两个单独的样式生成器网络，一个用于字体传输的统一平台，分别为统一的平台，另一个用于风格化和命运化。命令阶段首先提取字体参考图像的字体样式，然后字体传输阶段使用所需的字体样式生成目标内容。最后，样式阶段呈现有关参考图像中纹理样式的结果字体图像。此外，考虑到配对艺术文本图像的难度数据采集，我们的模型是在无监督的设置下设计的，可以从未配对的数据中有效地优化所有阶段。定性和定量结果是在艺术文本基准上执行的，这证明了我们提出的模型的出色性能。带有模型的代码将来将公开使用。

最近关于3D密集标题和视觉接地的研究取得了令人印象深刻的结果。尽管这两个方面都有发展，但可用的3D视觉语言数据的有限量导致3D视觉接地和3D密度标题方法的过度问题。此外，尚未完全研究如何辨别地描述复杂3D环境中的对象。为了解决这些挑战，我们呈现D3Net，即最终的神经扬声器 - 侦听器架构，可以检测，描述和辨别。我们的D3Net以自我批评方式统一3D密集的标题和视觉接地。D3Net的这种自我关键性质还引入了对象标题生成过程中的可怜性，并且可以通过部分注释的描述启用对Scannet数据的半监督培训。我们的方法在扫描带数据集的两个任务中优于SOTA方法，超越了SOTA 3D密度标题方法，通过显着的余量（23.56％的填充剂@ 0.5iou改进）。

In this paper, we propose a robust 3D detector, named Cross Modal Transformer (CMT), for end-to-end 3D multi-modal detection. Without explicit view transformation, CMT takes the image and point clouds tokens as inputs and directly outputs accurate 3D bounding boxes. The spatial alignment of multi-modal tokens is performed implicitly, by encoding the 3D points into multi-modal features. The core design of CMT is quite simple while its performance is impressive. CMT obtains 73.0% NDS on nuScenes benchmark. Moreover, CMT has a strong robustness even if the LiDAR is missing. Code will be released at https://github.com/junjie18/CMT.

Given the increasingly intricate forms of partial differential equations (PDEs) in physics and related fields, computationally solving PDEs without analytic solutions inevitably suffers from the trade-off between accuracy and efficiency. Recent advances in neural operators, a kind of mesh-independent neural-network-based PDE solvers, have suggested the dawn of overcoming this challenge. In this emerging direction, Koopman neural operator (KNO) is a representative demonstration and outperforms other state-of-the-art alternatives in terms of accuracy and efficiency. Here we present KoopmanLab, a self-contained and user-friendly PyTorch module of the Koopman neural operator family for solving partial differential equations. Beyond the original version of KNO, we develop multiple new variants of KNO based on different neural network architectures to improve the general applicability of our module. These variants are validated by mesh-independent and long-term prediction experiments implemented on representative PDEs (e.g., the Navier-Stokes equation and the Bateman-Burgers equation) and ERA5 (i.e., one of the largest high-resolution data sets of global-scale climate fields). These demonstrations suggest the potential of KoopmanLab to be considered in diverse applications of partial differential equations.

Rankings are widely collected in various real-life scenarios, leading to the leakage of personal information such as users' preferences on videos or news. To protect rankings, existing works mainly develop privacy protection on a single ranking within a set of ranking or pairwise comparisons of a ranking under the $\epsilon$-differential privacy. This paper proposes a novel notion called $\epsilon$-ranking differential privacy for protecting ranks. We establish the connection between the Mallows model (Mallows, 1957) and the proposed $\epsilon$-ranking differential privacy. This allows us to develop a multistage ranking algorithm to generate synthetic rankings while satisfying the developed $\epsilon$-ranking differential privacy. Theoretical results regarding the utility of synthetic rankings in the downstream tasks, including the inference attack and the personalized ranking tasks, are established. For the inference attack, we quantify how $\epsilon$ affects the estimation of the true ranking based on synthetic rankings. For the personalized ranking task, we consider varying privacy preferences among users and quantify how their privacy preferences affect the consistency in estimating the optimal ranking function. Extensive numerical experiments are carried out to verify the theoretical results and demonstrate the effectiveness of the proposed synthetic ranking algorithm.

Due to their ability to offer more comprehensive information than data from a single view, multi-view (multi-source, multi-modal, multi-perspective, etc.) data are being used more frequently in remote sensing tasks. However, as the number of views grows, the issue of data quality becomes more apparent, limiting the potential benefits of multi-view data. Although recent deep neural network (DNN) based models can learn the weight of data adaptively, a lack of research on explicitly quantifying the data quality of each view when fusing them renders these models inexplicable, performing unsatisfactorily and inflexible in downstream remote sensing tasks. To fill this gap, in this paper, evidential deep learning is introduced to the task of aerial-ground dual-view remote sensing scene classification to model the credibility of each view. Specifically, the theory of evidence is used to calculate an uncertainty value which describes the decision-making risk of each view. Based on this uncertainty, a novel decision-level fusion strategy is proposed to ensure that the view with lower risk obtains more weight, making the classification more credible. On two well-known, publicly available datasets of aerial-ground dual-view remote sensing images, the proposed approach achieves state-of-the-art results, demonstrating its effectiveness. The code and datasets of this article are available at the following address: https://github.com/gaopiaoliang/Evidential.

A noisy training set usually leads to the degradation of the generalization and robustness of neural networks. In this paper, we propose a novel theoretically guaranteed clean sample selection framework for learning with noisy labels. Specifically, we first present a Scalable Penalized Regression (SPR) method, to model the linear relation between network features and one-hot labels. In SPR, the clean data are identified by the zero mean-shift parameters solved in the regression model. We theoretically show that SPR can recover clean data under some conditions. Under general scenarios, the conditions may be no longer satisfied; and some noisy data are falsely selected as clean data. To solve this problem, we propose a data-adaptive method for Scalable Penalized Regression with Knockoff filters (Knockoffs-SPR), which is provable to control the False-Selection-Rate (FSR) in the selected clean data. To improve the efficiency, we further present a split algorithm that divides the whole training set into small pieces that can be solved in parallel to make the framework scalable to large datasets. While Knockoffs-SPR can be regarded as a sample selection module for a standard supervised training pipeline, we further combine it with a semi-supervised algorithm to exploit the support of noisy data as unlabeled data. Experimental results on several benchmark datasets and real-world noisy datasets show the effectiveness of our framework and validate the theoretical results of Knockoffs-SPR. Our code and pre-trained models will be released.

Temporal sentence grounding (TSG) aims to identify the temporal boundary of a specific segment from an untrimmed video by a sentence query. All existing works first utilize a sparse sampling strategy to extract a fixed number of video frames and then conduct multi-modal interactions with query sentence for reasoning. However, we argue that these methods have overlooked two indispensable issues: 1) Boundary-bias: The annotated target segment generally refers to two specific frames as corresponding start and end timestamps. The video downsampling process may lose these two frames and take the adjacent irrelevant frames as new boundaries. 2) Reasoning-bias: Such incorrect new boundary frames also lead to the reasoning bias during frame-query interaction, reducing the generalization ability of model. To alleviate above limitations, in this paper, we propose a novel Siamese Sampling and Reasoning Network (SSRN) for TSG, which introduces a siamese sampling mechanism to generate additional contextual frames to enrich and refine the new boundaries. Specifically, a reasoning strategy is developed to learn the inter-relationship among these frames and generate soft labels on boundaries for more accurate frame-query reasoning. Such mechanism is also able to supplement the absent consecutive visual semantics to the sampled sparse frames for fine-grained activity understanding. Extensive experiments demonstrate the effectiveness of SSRN on three challenging datasets.